Talking Digital Cities: Connecting Heterogeneous Digital Cities Via the Universal Mobile Interface

This paper introduces the universal mobile interface that allows users to navigate themselves into multiple heterogeneous digital cites with a single interface. The universal mobile interface connects a map-based information search system, an image-based virtual city space, and a 3D virtual city space, that had been developed independently. A user can walk-through the above systems simultaneously, while those systems provide the user with audio descriptions of the scenes. The aims of developing the universal mobile interface are: 1) to provide a single and transparent interface that connects heterogeneous digital cities, 2) to provide audio descriptions of scenes using characteristics of each virtual city space. 1 Why the Universal Mobile Interface? 1.1 Digital City Systems with Various Media Digital cities have been developed all over the world. Among them are efforts to make a virtual city where users can walkthrough, such as a 3D model of a real city, 2D map of a city, digital photo libraries of a city, and so on [2]. As computers are equipped with more power with cheaper cost, more and more digital cities that try to represent a real city will be developed. Moreover, recent rapid growth of digital camera market would increase the motivation of creating a digital city on the Internet. However, those digital cities which aim is to provide a walkthrough are developed independently. There is no common design to develop such digital cities for developers. At the same time, there is no common interface for users to walkthrough/manipulate those digital cities. If there is an interface that integrates heterogeneous digital cities that represent the same city, users can make use of the existing digital city from different perspectives. Digital cities are developed to provide regional information [2]. Such regional information is also necessary when a user is not at desk sitting in front of a computer. However, a precise 3D model of a real city would not be displayed on a small PDA or a mobile phone. It order to make full use of the existing digital cities, we need to provide “carriable” digital cities that give regional information to a user. 1.2 The Goals of the Universal Mobile Interface The aim of creating the universal mobile interface is to connect digital cities that are not originally designed to be used with other digital cities that represents the same area. The universal mobile interface should provide a user to navigate oneself into multiple digital cities through a single interface. The universal mobile interface has the following goals: 1. Single and simple interface: The universal mobile interface provides a single interface to multiple digital city systems so that a user can navigate oneself without learning specific commands to operate each system. The manipulation or movements within the city spaces should be intuitive and simple. Thus the universal mobile interface supports three input devices, namely, arrow keys, a joystick, and voice commands to move around the virtual spaces connected to the universal mobile interface. 2. Shared location: The universal mobile interface provides a simultaneous walkthrough experience of multiple digital cities. The universal mobile interface has its own coordinate system. The user’s current location (coordinates) is converted to other coordinates that correspond to other virtual spaces. A mobile user with GPS can move around virtual spaces via the universal mobile interface. Detail description of the system is made in the later chapter. 3. Text based information: The universal mobile interface provides text-based information based on the user’s current location by using text annotations attached to objects in the virtual spaces. The texts are converted to audio descriptions using text-to-speech. Thus users with vision problems or mobile users who do not have a display can receive textual information from multiple virtual spaces. 4. Prioritizing location based text information: The universal mobile interface works as a filter to provide most useful text information for the user. Each virtual space has an initial priority value for giving text information to the user. The user can change each priority value according to the user’s preferences. 2 Existing Digital City Systems This section describes the existing digital cities that have been developed for the research project called “Universal Design of Digital City.” Those digital cities represent the city of Kyoto from a different perspective and technology. Each city was developed independently. 2.1 Image-based Virtual City: TownDigitizing TownDigitizing bridges the digital and physical cities by using computer vision technologies [5]. TownDigitizing is a technology that forms a real time virtual space that permits walk-through. The images are captured by an omni-directional camera that takes 360-degree images of the real city. The images are automatically fused based on the visitor’s virtual location to reproduce what the user would see in a corresponding physical city. Fig.1. shows a TownDigitized version of downtown Kyoto. The newer version of TownDigtizing, that captures live images with omni-directional cameras installed inside a city and creates an image based virtual space automatically, is currently developed. Fig.1. Image based virtual city of Kyoto using TownDigitizing 2.2 3D VRML Model of Downtown Kyoto: FreeWalk FreeWalk provides a precise 3D VRML model of a real city [3]. Users enter into the virtual space via computer networks as avatars to navigate through the space and communicate with other avatars. Fig. 2. shows a 3D VRML model of downtown Kyoto using FreeWalk. Fig.2. 3D VRML model of Kyoto city by FreeWalk 2.3 Map-based Geographical Information System: Kyoto SEARCH KyotoSEARCH is a local information searching system where human cognitive map is reconstructed based on web resources [4]. One main aim of KyotoSEARCH is to analyze landmarks of a city. It includes extraction of landmarks, calculation of influences and mutual effects between landmarks, and characterization of each landmark. Fig. 3. shows a map of downtown Kyoto using KyotoSEARCH. Fig.3. Map of Kyoto city by KyotoSEARCH 2.4 Characteristics of Information from Digital Cities The above three virtual cities can provide specific information according to the nature of the technology used to develop the digital cities. The 3D model with FreeWalk is suitable for providing static and detailed information of buildings and shops, such as their size and colors. However, developing a detailed 3D model is costly and time-consuming, and requires technical knowledge. Avoiding obstacles while walking through a 3D city with a joystick or cursor keys may be difficult for a novice user. The image-based virtual city with TownDigitizing can provide lively scenes of the real city at relatively low cost and short development period without a high technical skill. Live cameras are useful for providing information on mobile entities of the real city, such as pedestrians and traffic on the road. However, walkable routes are limited to the places where photos of the real city are available. Users of a image-based virtual city cannot walk freely into the space. The above systems, namely, 3D model of a real city and a image-based virtual city are useful to provide detailed micro view from a user’s location, while the map based KyotoSEARCH can provide geographical and directional information by providing a macro view of the user’s location. However, a map-based system lacks visual image of the real city. Table 1. shows a comparison of the digital city characteristics described above. Comparison of dimensions is described in chapter 3. The universal mobile interface aims to provide combined city information by strengthening and complementing the characteristics of those virtual spaces each other. Table 1. Comparison of characteristics of digital cities that use 3D model, photo-realistic images, and 2D scalable map. 3D virtual spaces Image-based city spaces Map-based city information spaces Feature Detailed 3D model Photo-realistic images Scalable 2D map World Dimension 3D 2-2.5D 2D User space dimension 2D 2D 2D Movable space Anywhere except obstacles (walls etc.) Along pre-defined routes Anywhere Interactivity Very high High Low Macro view High Not provided Very high Micro view Very high High N/A Real-time information Possible but expensive Possible with live camera, relatively easy Possible (GIS) Development Cost Very high Relatively low Low 2.5 Scenario of Using the Universal Mobile Interface Typical scenarios of using the universal mobile interface are as follows: Scenario for a fixed user A senior person or computer novice is accessing a virtual city of Kyoto via the universal mobile interface. He does not know how to use a mouse or keyboard. He selects a location where he wants to start exploring the city by telling the system “Shijyo-kawaramachi.” He uses voice command to move forward, turn right, stop and so on to navigate himself into the space. The system presents the 3D model, images and map of his current location, while explaining shop names, their specialties, etc. The 3D virtual space tells the user, “The brown-colored shop on your left sells tea utensils.” The image-based system shows a live image of the shop. The map system adds, “The shop is located at 50m south from the ShijoKawaramachi intersection.” The universal mobile interface provides three different views and audio descriptions of the same place. Scenario for a mobile user A first-time visitor of downtown Kyoto is walking around the city with a mobile phone. The mobile phone is equipped with GPS and can run Java applications on it. The mobile phone is connected to multiple digital cities of Kyoto via the universal mobile interface. As the user moves forward along a street, the universal mobile interface gives audio information on landmarks ahead, shops to be visited, or which way the street is leading, etc. The 3D virtual space gives the user detailed information on each landmark, such as “The brown building on your right is a famous bakery.” The map system gives directional information such as “If you keep going along this street to the west for 1.6 kilometers, there is another bakery on your right.” The image-based virtual space gives lively information to the user, such as “There is a long line in front of the bakery ahead of you.” The user can change priority values that are set to each virtual space by giving feedback to each virtual space while it is giving audio description to the user, such as “That sounds interesting.” (while the 3D virtual space is giving information on the bakery). Then the priority value of the 3D virtual space increases so that it can give information more often than other virtual spaces. The mechanism of achieving the above scenario is described in the following section. 3 System Description This section gives detailed system description of each component that realizes the goal of the universal mobile interface described in the section 1.2. 3.1 Single and Simple Interface The universal mobile interface is a windows-based application that communicates with various digital city systems via TCP/IP. A user can connect to digital cities via the universal mobile interface by selecting a specific geographical point (i.e., an intersection, a department store, etc.), or by using a longitude and latitude of a geographical point. A user can move around the connected digital cities with arrow keys, joystick, or voice commands. Fig. 4. shows a fixed user using the universal mobile interface to connect to a image-based virtual space, 3D VRML model, and 2D map of downtown Kyoto. The universal mobile interface shows visual city information in addition to audio descriptions. All three digital cities display the same location. Fig. 4. A fixed user is using the universal mobile interface (visual information provided as a service monitor, audio description is provided) to connect to a image-based virtual space, 3D VRML model, and 2D map of downtown Kyoto. All three digital cities display the same location and move synchronously. 3.2 Location and Movement Control –Shared Location To achieve synchronous location and movement control among various digital cities, the universal mobile interface has its own coordinate system. When a user moves around the connected digital cities, the user actually moves within the coordinate system of the universal mobile interface. Fig.5. shows the diagram of location and movement control provided by the universal mobile interface. Key/Joystick operation Audio command GPS Speech recognition tool GPS input handling Key/Joystick operation handling Movement control (determines orientation and distance for the next movement) Movement execution (calculate the next coordinates/update the user’s coordinates) coordinate conversion coordinate conversion coordinate conversion 3D space Map-based space U er loation m aagem nt Latitude, longitude, altitude Orientation and distance Orientation and distance Orientation, distance and coordinates Updated user’s coordinates in UI coordinate system New coordinates New coordinates New coordinates Updated user’s coordinates in UI coordinate system Updated user’s coordinates in UI coordinate system Updated location Current location Movement result